Image Pairs Research Articles

Deep learning-based whole-body PSMA PET/CT attenuation correction utilizing Pix-2-Pix GAN.

Sequential PET/CT studies oncology patients can undergo during their treatment follow-up course is limited by radiation dosage. We propose an artificial intelligence (AI) tool to produce attenuation-corrected PET (AC-PET) images from non-attenuation-corrected PET (NAC-PET) images to reduce need for low-dose CT scans. A deep learning algorithm based on 2D Pix-2-Pix generative adversarial network (GAN) architecture was developed from paired AC-PET and NAC-PET images. 18F-DCFPyL PSMA PET-CT studies from 302 prostate cancer patients, split into training, validation, and testing cohorts (n = 183, 60, 59, respectively). Models were trained with two normalization strategies: Standard Uptake Value (SUV)-based and SUV-Nyul-based. Scan-level performance was evaluated by normalized mean square error (NMSE), mean absolute error (MAE), structural similarity index (SSIM), and peak signal-to-noise ratio (PSNR). Lesion-level analysis was performed in regions-of-interest prospectively from nuclear medicine physicians. SUV metrics were evaluated using intraclass correlation coefficient (ICC), repeatability coefficient (RC), and linear mixed-effects modeling. Median NMSE, MAE, SSIM, and PSNR were 13.26%, 3.59%, 0.891, and 26.82, respectively, in the independent test cohort. ICC for SUVmax and SUVmean were 0.88 and 0.89, which indicated a high correlation between original and AI-generated quantitative imaging markers. Lesion location, density (Hounsfield units), and lesion uptake were all shown to impact relative error in generated SUV metrics (all p < 0.05). The Pix-2-Pix GAN model for generating AC-PET demonstrates SUV metrics that highly correlate with original images. AI-generated PET images show clinical potential for reducing the need for CT scans for attenuation correction while preserving quantitative markers and image quality.

A road surface reconstruction dataset for autonomous driving

Recent developments in intelligent robot systems, especially autonomous vehicles, put forward higher requirements for safety and comfort. Road conditions are crucial factors affecting the comprehensive performance of ground vehicles. Nonetheless, existing environment perception datasets for autonomous driving lack attention to road surface areas. In this paper, we introduce the road surface reconstruction dataset, providing multi-modal, high-resolution, and high-precision data collected by real-vehicle platform in diverse driving conditions. It covers common road types containing approximately 16,000 pairs of stereo images, point clouds, and ground-truth depth/disparity maps, with accurate data processing pipelines to ensure its quality. Preliminary evaluations reveal the effectiveness of our dataset and the challenge of the task, underscoring substantial opportunities of it as a valuable resource for advancing computer vision techniques. The reconstructed road structure and texture contribute to the analysis and prediction of vehicle responses for motion planning and control systems.

A New Distance Measurement Method for Perceptual Image Hashing

A large number of perceptual image hashing schemes have been developed in recent decades, and they usually calculated the hash distances using Euclidean distance, correlation coefficient, normalized Hamming distance, etc. These hash distances are not customized measurements for hash sequences, which are all single values. To this end, a new distance measurement method, i.e. a distance vector instead of a single value, is proposed in this paper. Since a hash sequence can be regarded as the compressed representation of image perceptual content, the correlation between images is maintained between hash sequences. In our method, the correlation between hash sequences is captured by residuals and co-occurrence matrix to form a distance vector. The correlation differences between perceptually similar and distinct images can be captured in the obtained distance vectors. Using a binary classifier, it is easy to classify the distance vectors of perceptually similar image pairs and perceptually distinct image pairs. As a result, content authentication accuracy of perceptual image hashing can be effectively increased, which is verified by experimental results.

Abstract PO1-07-07: Radiomics features for distinguishing true recurrence versus new primary tumor following breast-conserving treatment

Abstract Background Despite modern surgical and irradiation techniques, ipsilateral breast tumor recurrence (IBTR) accounts for 5-15% of all cancer recurrences in women treated with breast conservative treatment. Historically, the methods to define true recurrence (TR) and new primary tumor (NP) for IBTR mainly rely on clinical and pathological criteria, limiting the accuracy of the discerption and causing misclassification. This study aimed to develop a preoperative, noninvasive model for distinguishing IBTR by integrating clinicopathological review with paired dynamic contrast-enhanced breast magnetic resonance imaging (DCE-MRI) at diagnosis and following in-breast recurrence. MethodsWe retrospectively extracted radiomics features from MRI to develop a radiomic cohort of IBTR (n =46), among which all patients underwent paired preoperative DCE-MRI. This radiomic cohort was divided into a training cohort (n =27) and a validation cohort (n =19) with stratified random sampling. Classification of IBTR as TR or NP on the basis of tumor location, histologic subtype, estrogen receptor, and HER2 status was set as the gold standard. The least absolute shrinkage and selection operator (LASSO) regression and logistic regression were utilized to perform radiomics feature selection and model training, respectively. The clinical utility of the model was determined via decision curve analysis (DCA). Results We selected three radiomics features (first-order feature Kurtosis from the first postcontrast phase on IBTR DCE-MRI, GLCM feature Imc2 from the first postcontrast phase on IBTR DCE-MRI, and the delta feature Correlation from the precontrast phase of the change between IBTR and primary tumor DCE-MRI) to develop an IBTR-classification predicting radiomic model, which performed well in the validation cohort (AUC 0.867, 95% confidence interval (CI) 0.694-1). Further investigation for sensitivity (73.3%) and specificity (100%) verified a favorable concordance between the radiomic classification and the conventional standard, with a diagnostic accuracy of 79%. Conclusions: Our study demonstrated the feasibility of the radiomics model in predicting IBTR classification and provided preoperative information about the nature of “recurrence”. This might have important implications in surgical approaches and multidisciplinary care for IBTR. Further efforts are needed to improve the reproducibility of radiomics features and models in multiple centers. Citation Format: Feilin Qu, Guan-Hua Su, Jin-Hui Li, Chao You, Zhi-Ming Shao. Radiomics features for distinguishing true recurrence versus new primary tumor following breast-conserving treatment [abstract]. In: Proceedings of the 2023 San Antonio Breast Cancer Symposium; 2023 Dec 5-9; San Antonio, TX. Philadelphia (PA): AACR; Cancer Res 2024;84(9 Suppl):Abstract nr PO1-07-07.

Stereo matching from monocular images using feature consistency

AbstractSynthetic images facilitate stereo matching. However, synthetic images may suffer from image distortion, domain bias, and stereo mismatch, which would significantly restrict the widespread use of stereo matching models in the real world. The first goal in this paper is to synthesize real‐looking images for minimizing the domain bias between the synthesized and real images. For this purpose, sharpened disparity maps are produced from a mono real image. Then, stereo image pairs are synthesized using these imperfect disparity maps and the single real image in the proposed pipeline. Although the synthesized images are as realistic as possible, the domain styles of the synthesized images are always very different from the real images. Thus, the second goal is to enhance the domain generalization ability of the stereo matching network. For that, the feature extraction layer is replaced with a teacher–student model. Then, a constraint of binocular contrast features is imposed on the output of the model. When tested on the KITTI, ETH3D, and Middlebury datasets, the accuracy of the method outperforms traditional methods by at least 30%. Experiments demonstrate that the approaches are general and can be conveniently embedded into existing stereo networks.

Enhancing Power Lines Detection Using Deep Learning and Feature-Level Fusion of Infrared and Visible Light Images

AbstractThe detection of power lines is critical for flight safety, especially for drones and low-flying aircraft. Power line detection models help prevent collisions, reducing potential damage and preserving lives, while also safeguarding critical infrastructure. This has led to significant research efforts to develop accurate detection models. In this study, we employ paired infrared–visible power line datasets to train three distinct deep learning models. The first two models are sequential deep learning models based on VGG16 and AlexNet networks. They are tailored for detection in visible images, while they were optimized again for infrared images. For the third model, we introduce an innovative deep learning architecture utilizing Functional Application Programming Interface, affording us the flexibility to construct a multi-input model with shared layers. Our proposed model accepts paired images (visible and infrared) as inputs. Then, a feature-level fusion process is applied to merge the extracted features from both inputs and generate an enriched feature map. This approach amalgamates the advantages of visible images, which boast high resolution and rich texture features, with infrared images, which excel in high contrast and clear vision under adverse environmental conditions. Comparing the outcomes of the three models, our proposed model emerges as the front runner, boasting an impressive accuracy rate of 99.37%. Moreover, real-time processing was adopted by conducting ablation experiments to optimize the model and reduce the number of trainable parameters, resulting in an inference speed of 2.7 milliseconds per frame.

Enhancing Visual Grounding in Vision-Language Pre-Training With Position-Guided Text Prompts.

Vision-Language Pre-Training (VLP) has demonstrated remarkable potential in aligning image and text pairs, paving the way for a wide range of cross-modal learning tasks. Nevertheless, we have observed that VLP models often fall short in terms of visual grounding and localization capabilities, which are crucial for many downstream tasks, such as visual reasoning. In response, we introduce a novel Position-guided Text Prompt (PTP) paradigm to bolster the visual grounding abilities of cross-modal models trained with VLP. In the VLP phase, PTP divides an image into N x N blocks and employs a widely-used object detector to identify objects within each block. PTP then reframes the visual grounding task as a fill-in-the-blank problem, encouraging the model to predict objects in given blocks or regress the blocks of a given object, exemplified by filling "[P]" or "[O]" in a PTP sentence such as "The block [P] has a [O]." This strategy enhances the visual grounding capabilities of VLP models, enabling them to better tackle various downstream tasks. Additionally, we integrate the seconda-order relationships between objects to further enhance the visual grounding capabilities of our proposed PTP paradigm. Incorporating PTP into several state-of-the-art VLP frameworks leads to consistently significant improvements across representative cross-modal learning model architectures and multiple benchmarks, such as zero-shot Flickr30 k Retrieval (+5.6 in average recall@1) for ViLT baseline, and COCO Captioning (+5.5 in CIDEr) for the state-of-the-art BLIP baseline. Furthermore, PTP attains comparable results with object-detector-based methods and a faster inference speed, as it discards its object detector during inference, unlike other approaches.

Affine registration of thermal images of plantar feet using convolutional neural networks

The use of a thermal camera to detect abnormal plantar foot temperature changes can be an effective way to identify the early signs of diabetic foot ulceration. In this work, we performed the affine registration of the plantar foot thermal images using four models based on convolutional neural networks. The process include two parts: an affine registration model for estimating transformation parameters and a spatial transformer for getting the registered image. The performances of the four models were evaluated using the Dice similarity coefficient (DSC), Mean Square Error (MSE), and peak signal-to-noise ratio (PSNR). In the first step, Methods were applied to register the left and right feet of the same subject, called “contralateral registration” and in the second step, the methods were evaluated on a pair of images of the same subject taken in two different times (T0 and T10) using a cold stress test protocol. Results showed that the used convolutional neural networks are robust in both types of registration (contralateral and multitemporal), and they are suitable for the targeted application, with the DSC of 95% for contralateral registration and a DSC of 92% for multitemporal registration. Furthermore, a transversal clinical study was perform on diabetic patients, that classified individuals into ischemic and non-ischemic groups. The objective was to analyze the coherence between the thermal results and medical data. The mean absolute point-to-point temperature difference |ΔT| between left and right foot is lower in non-ischemic patients than in those with ischemia, with p<0.05.

Self-Training Boosted Multi-Factor Matching Network for Composed Image Retrieval.

The composed image retrieval (CIR) task aims to retrieve the desired target image for a given multimodal query, i.e., a reference image with its corresponding modification text. The key limitations encountered by existing efforts are two aspects: 1) ignoring the multiple query-target matching factors; 2) ignoring the potential unlabeled reference-target image pairs in existing benchmark datasets. To address these two limitations is non-trivial due to the following challenges: 1) how to effectively model the multiple matching factors in a latent way without direct supervision signals; 2) how to fully utilize the potential unlabeled reference-target image pairs to improve the generalization ability of the CIR model. To address these challenges, in this work, we first propose a CLIP-Transformer based muLtI-factor Matching Network (LIMN), which consists of three key modules: disentanglement-based latent factor tokens mining, dual aggregation-based matching token learning, and dual query-target matching modeling. Thereafter, we design an iterative dual self-training paradigm to further enhance the performance of LIMN by fully utilizing the potential unlabeled reference-target image pairs in a weakly-supervised manner. Specifically, we denote the iterative dual self-training paradigm enhanced LIMN as LIMN+. Extensive experiments on four datasets, including FashionIQ, Shoes, CIRR, and Fashion200 K, show that our proposed LIMN and LIMN+ significantly surpass the state-of-the-art baselines.

An unsupervised multi-focus image fusion method via. dual-channel convolutional network and discriminator

The challenge in multi-focus image fusion tasks lies in accurately preserving the complementary information from the source images in the fused image. However, existing datasets often lack ground truth images, making it difficult for some full-reference loss functions (such as SSIM) to effectively participate in model training, thereby further affecting the performance of retaining source image details. To address this issue, this paper proposes an unsupervised dual-channel dense convolutional method, DCD, for multi-focus image fusion. DCD designs Patch processing blocks specifically for the fusion task, which segment the source image pairs into equally sized patches and evaluate their information to obtain a reconstructed image and a set of adaptive weight coefficients. The reconstructed image is used as the reference image, enabling unsupervised methods to utilize full-reference loss functions in training and overcoming the challenge of lacking labeled data in the training set.Furthermore, considering that the human visual system (HVS) is more sensitive to brightness than color, DCD trains the dual-channel network using both RGB images and their luminance components. This allows the network to focus more on the brightness information while preserving the color and gradient details of the source images, resulting in fused images that are more compatible with the HVS. The adaptive weight coefficients obtained through the Patch processing blocks are also used to determine the degree of preservation of the brightness information in the source images.Finally, comparative experiments on different datasets also demonstrate the superior performance of DCD in terms of fused image quality compared to other methods.

Self-Supervised Joint Learning for pCLE Image Denoising

Probe-based confocal laser endoscopy (pCLE) has emerged as a powerful tool for disease diagnosis, yet it faces challenges such as the formation of hexagonal patterns in images due to the inherent characteristics of fiber bundles. Recent advancements in deep learning offer promise in image denoising, but the acquisition of clean-noisy image pairs for training networks across all potential scenarios can be prohibitively costly. Few studies have explored training denoising networks on such pairs. Here, we propose an innovative self-supervised denoising method. Our approach integrates noise prediction networks, image quality assessment networks, and denoising networks in a collaborative, jointly trained manner. Compared to prior self-supervised denoising methods, our approach yields superior results on pCLE images and fluorescence microscopy images. In summary, our novel self-supervised denoising technique enhances image quality in pCLE diagnosis by leveraging the synergy of noise prediction, image quality assessment, and denoising networks, surpassing previous methods on both pCLE and fluorescence microscopy images.

Iterative reconstruction for limited-angle CT using implicit neural representation

Objective. Limited-angle computed tomography (CT) presents a challenge due to its ill-posed nature. In such scenarios, analytical reconstruction methods often exhibit severe artifacts. To tackle this inverse problem, several supervised deep learning-based approaches have been proposed. However, they are constrained by limitations such as generalization issue and the difficulty of acquiring a large amount of paired CT images. Approach. In this work, we propose an iterative neural reconstruction framework designed for limited-angle CT. By leveraging a coordinate-based neural representation, we formulate tomographic reconstruction as a convex optimization problem involving a deep neural network. We then employ differentiable projection layer to optimize this network by minimizing the discrepancy between the predicted and measured projection data. In addition, we introduce a prior-based weight initialization method to ensure the network starts optimization with an informed initial guess. This strategic initialization significantly improves the quality of iterative reconstruction by stabilizing the divergent behavior in ill-posed neural fields. Our method operates in a self-supervised manner, thereby eliminating the need for extensive data. Main results. The proposed method outperforms other iterative and learning-based methods. Experimental results on XCAT and Mayo Clinic datasets demonstrate the effectiveness of our approach in restoring anatomical features as well as structures. This finding was substantiated by visual inspections and quantitative evaluations using NRMSE, PSNR, and SSIM. Moreover, we conduct a comprehensive investigation into the divergent behavior of iterative neural reconstruction, thus revealing its suboptimal convergence when starting from scratch. In contrast, our method consistently produced accurate images by incorporating an initial estimate as informed initialization. Significance. This work showcases the feasibility to reconstruct high-fidelity CT images from limited-angle x-ray projections. The proposed methodology introduces a novel data-free approach to enhance medical imaging, holding promise across various clinical applications.

Comparative Analysis of Color Space and Channel, Detector, and Descriptor for Feature-Based Image Registration

The current study aimed to quantify the value of color spaces and channels as a potential superior replacement for standard grayscale images, as well as the relative performance of open-source detectors and descriptors for general feature-based image registration purposes, based on a large benchmark dataset. The public dataset UDIS-D, with 1106 diverse image pairs, was selected. In total, 21 color spaces or channels including RGB, XYZ, Y′CrCb, HLS, L*a*b* and their corresponding channels in addition to grayscale, nine feature detectors including AKAZE, BRISK, CSE, FAST, HL, KAZE, ORB, SIFT, and TBMR, and 11 feature descriptors including AKAZE, BB, BRIEF, BRISK, DAISY, FREAK, KAZE, LATCH, ORB, SIFT, and VGG were evaluated according to reprojection error (RE), root mean square error (RMSE), structural similarity index measure (SSIM), registration failure rate, and feature number, based on 1,950,984 image registrations. No meaningful benefits from color space or channel were observed, although XYZ, RGB color space and L* color channel were able to outperform grayscale by a very minor margin. Per the dataset, the best-performing color space or channel, detector, and descriptor were XYZ/RGB, SIFT/FAST, and AKAZE. The most robust color space or channel, detector, and descriptor were L*a*b*, TBMR, and VGG. The color channel, detector, and descriptor with the most initial detector features and final homography features were Z/L*, FAST, and KAZE. In terms of the best overall unfailing combinations, XYZ/RGB+SIFT/FAST+VGG/SIFT seemed to provide the highest image registration quality, while Z+FAST+VGG provided the most image features.

NRPerson: A Non-Registered Multi-Modal Benchmark for Tiny Person Detection and Localization

In recent years, the detection and localization of tiny persons have garnered significant attention due to their critical applications in various surveillance and security scenarios. Traditional multi-modal methods predominantly rely on well-registered image pairs, necessitating the use of sophisticated sensors and extensive manual effort for registration, which restricts their practical utility in dynamic, real-world environments. Addressing this gap, this paper introduces a novel non-registered multi-modal benchmark named NRPerson, specifically designed to advance the field of tiny person detection and localization by accommodating the complexities of real-world scenarios. The NRPerson dataset comprises 8548 RGB-IR image pairs, meticulously collected and filtered from 22 video sequences, enriched with 889,207 high-quality annotations that have been manually verified for accuracy. Utilizing NRPerson, we evaluate several leading detection and localization models across both mono-modal and non-registered multi-modal frameworks. Furthermore, we develop a comprehensive set of natural multi-modal baselines for the innovative non-registered track, aiming to enhance the detection and localization of unregistered multi-modal data using a cohesive and generalized approach. This benchmark is poised to facilitate significant strides in the practical deployment of detection and localization technologies by mitigating the reliance on stringent registration requirements.

Uncertain prediction of deformable image registration on lung CT using multi-category features and supervised learning.

The assessment of deformable registration uncertainty is an important task for the safety and reliability of registration methods in clinical applications. However, it is typically done by a manual and time-consuming procedure. We propose a novel automatic method to predict registration uncertainty based on multi-category features and supervised learning. Three types of features, including deformation field statistical features, deformation field physiologically realistic features, and image similarity features, are introduced and calculated to train the random forest regressor for local registration uncertain prediction. Deformation field statistical features represent the numerical stability of registration optimization, which are correlated to the uncertainty of deformation fields; deformation field physiologically realistic features represent the biomechanical properties of organ motions, which mathematically reflect the physiological reality of deformation; image similarity features reflect the similarity between the warped image and fixed image. The multi-category features comprehensively reflect the registration uncertainty. The strategy of spatial adaptive random perturbations is also introduced to accurately simulate spatial distribution of registration uncertainty, which makes deformation field statistical features more discriminative to the uncertainty of deformation fields. Experiments were conducted on three publicly available thoracic CT image datasets. Seventeen randomly selected image pairs are used to train the random forest model, and 9 image pairs are used to evaluate the prediction model. The quantitative experiments on lung CT images show that the proposed method outperforms the baseline method for uncertain prediction of classical iterative optimization-based registration and deep learning-based registration with different registration qualities. The proposed method achieves good performance for registration uncertain prediction, which has great potential in improving the accuracy of registration uncertain prediction.

Cherry growth modeling based on Prior Distance Embedding contrastive learning: Pre-training, anomaly detection, semantic segmentation, and temporal modeling

In current plant phenotyping research, the study of plant time-series images based on deep learning has received widespread attention. While such image data is relatively easy to obtain, the cost of annotation is high. One efficient method for achieving cost-effective training is through contrastive learning. Plant growth is slow, and the changes in image sequences over a period of time are small, with simple semantic information. Previous contrastive pre-training models struggled to effectively distinguish positive samples from the same image with different augmented views and similar negative samples from different images. Therefore, this paper proposes a method called self-supervised contrastive learning method for plant time-series images with a Priori Distance Embedding (PDE). The semantic information in images corresponding to different phenological stages of plants varies. This method transforms this crucial domain knowledge into prior distances for image pairs and conducts contrastive learning pre-training. The learned weights can be transferred to downstream tasks. Building upon this method, experiments were conducted on cherry time-series images to assess the quality of pre-training through a plant phenotyping image semantic segmentation task. To provide a comprehensive example of plant time-series image phenotypic analysis, this paper establishes a cherry growth temporal model, specifically including PDE pre-training, anomaly detection, semantic segmentation, and recording the results from the temporal dimension. The experiments indicate that this self-supervised contrastive learning method can be effectively applied to the pre-training of plant time-series images, demonstrating broad applicability in various computer vision studies related to plant phenotyping.

Enhancing Surveillance Vision with Multi-Layer Deep Learning Representation

This paper aimed to develop a method for generating sand–dust removal and dehazed images utilizing CycleGAN, facilitating object identification on roads under adverse weather conditions such as heavy dust or haze, which severely impair visibility. Initially, the study addressed the scarcity of paired image sets for training by employing unpaired CycleGAN training. The CycleGAN training module incorporates hierarchical single-scale Retinex (SSR) images with varying sigma sizes, facilitating multiple-scaled trainings. Refining the training data into detailed hierarchical layers for virtual paired training enhances the performance of CycleGAN training. Conventional sand–dust removal or dehazing algorithms, alongside deep learning methods, encounter challenges in simultaneously addressing sand–dust removal and dehazing with a singular algorithm. Such algorithms often necessitate resetting hyperparameters to process images from both scenarios. To overcome this limitation, we proposed a unified approach for removing sand–dust and haze phenomena using a single model, leveraging images processed hierarchically with SSR. The image quality and image sharpness metrics of the proposed method were BRIQUE, PIQE, CEIQ, MCMA, LPC-SI, and S3. In sand–dust environments, the proposed method achieved the highest scores, with an average of 21.52 in BRISQUE, 0.724 in MCMA, and 0.968 in LPC-SI compared to conventional methods. For haze images, the proposed method outperformed conventional methods with an average of 3.458 in CEIQ, 0.967 in LPC-SI, and 0.243 in S3. The images generated via this proposed method demonstrated superior performance in image quality and sharpness evaluation compared to conventional algorithms. The outcomes of this study hold particular relevance for camera images utilized in automobiles, especially in the context of self-driving cars or CCTV surveillance systems.

Deep-learning model for background parenchymal enhancement classification in contrast-enhanced mammography.

Breast Background Parenchymal Enhancement (BPE) is correlated with&#xD;the risk of breast cancer. BPE level is currently assessed by radiologists in Contrast-Enhanced&#xD;Mammography (CEM) using 4 classes: minimal, mild, moderate and marked, as described&#xD;in Breast Imaging Reporting and Data System (BI-RADS). However, BPE classification re-&#xD;mains subject to intra- and inter-reader variability. Fully automated methods to assess BPE&#xD;level have already been developed in breast Contrast-Enhanced MRI (CE-MRI) and have been&#xD;shown to provide accurate and repeatable BPE level classification. However, to our knowledge,&#xD;no BPE level classification tool is available in the literature for CEM.&#xD;Materials & Methods: A BPE level classification tool based on Deep Learning (DL) has&#xD;been trained and optimized on 7012 CEM image pairs (low-energy and recombined images)&#xD;and evaluated on a dataset of 1013 image pairs. The impact of image resolution, backbone&#xD;architecture and loss function were analyzed, as well as the influence of lesion presence and&#xD;type on BPE assessment. The evaluation of the model performance was conducted using dif-&#xD;ferent metrics including 4-class balanced accuracy and mean absolute error. The results of the&#xD;optimized model for a binary classification: minimal/mild versus moderate/marked, were also&#xD;investigated.&#xD;Results: The optimized model achieved a 4-class balanced accuracy of 71.5% (95% CI:&#xD;71.2-71.9) with 98.8% of classification errors between adjacent classes. For binary classifi-&#xD;cation, the accuracy reached 93.0%. A slight decrease in model accuracy is observed in the&#xD;presence of lesions, but it is not statistically significant, suggesting that our model is robust to&#xD;the presence of lesions in the image for a classification task. Visual assessment also confirms&#xD;that the model is more affected by non-mass enhancements than by mass-like enhancements.&#xD;Conclusion: The proposed BPE classification tool for CEM achieves similar results than&#xD;what is published in the literature for CE-MRI.

Multi-modal medical image fusion using improved dual-channel PCNN.

This paper proposes a medical image fusion method in the non-subsampled shearlet transform (NSST) domain to combine a gray-scale image with the respective pseudo-color image obtained through different imaging modalities. The proposed method applies a novel improved dual-channel pulse-coupled neural network (IDPCNN) model to fuse the high-pass sub-images, whereas the Prewitt operator is combined with maximum regional energy (MRE) to construct the fused low-pass sub-image. First, the gray-scale image and luminance of the pseudo-color image are decomposed using NSST to find the respective sub-images. Second, the low-pass sub-images are fused by the Prewitt operator and MRE-based rule. Third, the proposed IDPCNN is utilized to get the fused high-pass sub-images from the respective high-pass sub-images. Fourth, the luminance of the fused image is obtained by applying inverse NSST on the fused sub-images, which is combined with the chrominance components of the pseudo-color image to construct the fused image. A total of 28 diverse medical image pairs, 11 existing methods, and nine objective metrics are used in the experiment. Qualitative and quantitative fusion results show that the proposed method is competitive with and even outpaces some of the existing medical fusion approaches. It is also shown that the proposed method efficiently combines two gray-scale images.

OmniGlasses: an optical aid for stereo vision CNNs to enable omnidirectional image processing

Stereo vision is a key technology for 3D scene reconstruction from image pairs. Most approaches process perspective images from commodity cameras. These images, however, have a very limited field of view and only picture a small portion of the scene. In contrast, omnidirectional images, also known as fisheye images, exhibit a much larger field of view and allow a full 3D scene reconstruction with a small amount of cameras if placed carefully. However, omnidirectional images are strongly distorted which make the 3D reconstruction much more sophisticated. Nowadays, a lot of research is conducted on CNNs for omnidirectional stereo vision. Nevertheless, a significant gap between estimation accuracy and throughput can be observed in the literature. This work aims to bridge this gap by introducing a novel set of transformations, namely OmniGlasses. These are incorporated into the architecture of a fast network, i.e., AnyNet, originally designed for scene reconstruction on perspective images. Our network, Omni-AnyNet, produces accurate omnidirectional distance maps with a mean absolute error of around 13 cm at 48.4 fps and is therefore real-time capable.